Introducing the FMM Procedure for Finite Mixture Models

You’ve collected the data and performed a preliminary analysis with a linear regression. But the residuals have several modes, and transformations don’t help. You need a different approach, and that calls for the FMM procedure. PROC FMM fits finite mixture models, which enable you to describe your data with mixtures of different distributions so you can account for underlying heterogeneity and address overdispersion. PROC FMM offers a wide selection of continuous and discrete distributions, and it provides automated model selection to help you choose the number of components. Bayesian techniques are also available for many analyses. This paper provides an overview of the capabilities of the FMM procedure and illustrates them with applications drawn from a variety of fields. INTRODUCTION Most statistical methods assume that you have a sample of observations, all of which come from the same distribution, and that you are interested in modeling that one distribution. If you actually have data from more than one distribution with no information to identify which observation goes with which distribution, standard models won’t help you. However, finite mixture models might come to the rescue. They use a mixture of parametric distributions to model data, estimating both the parameters for the separate distributions and the probabilities of component membership for each observation. Finite mixture models provide a flexible framework for analyzing a variety of data. Suppose your objective is to describe the distribution of a response variable. If the corresponding data are multimodal, skewed, heavy-tailed, or exhibit kurtosis, they may not be representative of most known distributions. In this case, you often use a nonparametric method such as kernel density estimation to describe the distribution. A kernel density estimate generates a smoothed, numerical approximation to the unknown distribution function and estimates the distribution’s percentiles. Although this approach is useful, it might not be the most concise way to describe an unknown distribution. A finite mixture model provides a parametric alternative that describes the unknown distribution in terms of mixtures of known distributions. A finite mixture model also enables you to assess the probabilities of events or simulate draws from the unknown distribution the same way you do when your data are from a known distribution. Finite mixture models also provide a parametric modeling approach to one-dimensional cluster analysis. This approach uses the fitted component distributions and the estimated mixing probabilities to compute a posterior probability of component membership. An observation is assigned membership to the component with the maximum posterior probability. A benefit of using a model-based approach to clustering is that it permits estimation and hypothesis testing within the framework of standard statistical theory (McLachlan and Basford 1988). Finally, finite mixture models provide a mechanism that can account for unobserved heterogeneity in the data. Certain important classifications of the data (such as region, age group, or gender) are not always measured. These latent classification variables can introduce underdispersion, overdispersion, or heteroscedasticity in a traditional model. Finite mixture models overcome these problems through their more flexible form. FINITE MIXTURE MODELS Consider a data set that is composed of people’s body weights. Figure 1 presents the pooled data. The histogram indicates an asymmetric distribution with three modes. Figure 2 displays separate histograms for age group and gender. Each distribution is symmetric, with only one mode. 1 Statistics and Data Analysis SAS Global Forum 2012